Stochastic gradient descent (SGD) is a widely adopted iterative method for optimizing differentiable objective functions. In this paper, we propose and discuss a novel approach to scale up SGD in applications involving non-convex functions and large datasets. We address the bottleneck problem arising when using both shared and distributed memory. Typically, the former is bounded by limited computation resources and bandwidth whereas the latter suffers from communication overheads. We propose a unified distributed and parallel implementation of SGD (named DPSGD) that relies on both asynchronous distribution and lock-free parallelism. By combining two strategies into a unified framework, DPSGD is able to strike a better trade-off between local computation and communication. The convergence properties of DPSGD are studied for non-convex problems such as those arising in statistical modelling and machine learning. Our theoretical analysis shows that DPSGD leads to speed-up with respect to the number of cores and number of workers while guaranteeing an asymptotic convergence rate of \(O(1/\sqrt{T})\) given that the number of cores is bounded by \(T^{1/4}\) and the number of workers is bounded by \(T^{1/2}\) where T is the number of iterations. The potential gains that can be achieved by DPSGD are demonstrated empirically on a stochastic variational inference problem (Latent Dirichlet Allocation) and on a deep reinforcement learning (DRL) problem (advantage actor critic - A2C) resulting in two algorithms: DPSVI and HSA2C. Empirical results validate our theoretical findings. Comparative studies are conducted to show the performance of the proposed DPSGD against the state-of-the-art DRL algorithms.
Thermal properties of fossil fuel are the key fundamental characteristics, which can distinguish any compound as a potential fuel. The performance of diesel fuel blend along with stability and solubility parameter designs are evaluated. The results from the experimental study indicate that the increase in hydrogen peroxide (H2O2) amount enhances the cetane number of diesel fuel blend significantly. However, the calorific value decreases as compared to pure diesel fuel. All values performed well according to the ASTM D‐975 diesel testing method. The thermodynamics of the prepared fuel blends also revealed that substantial solubility and diesel/H2O2 blend stability are provided even at lower temperatures. Such blends can be used as a feasible replacement of pure diesel fuel. 相似文献
In this communication, we demonstrate the preparation of Ni nanoparticles-graphene hybrid film via simultaneous electrochemical reduction of graphene oxide and Ni2+ on conductive substrate for the first time. We further show the utilization of such hybrid film as an efficient oxygen evolution reaction electrocatalyst with highly catalytic activity and good durability. 相似文献
The physical and chemical characteristics of biomaterial surface and hydrogels can be altered by external stimuli, such as light irradiation, temperature changes, pH shifts, shear stress forces, electrical forces, and the addition of small chemical molecules. Such external stimulus-responsive biomaterials represent promising candidates that have been developed for the culture and differentiation of embryonic stem (ES) cells, induced pluripotent stem (iPS) cells, and adult stem cells. Biomaterials that are designed to respond in a reversible manner to specific external signals can be formed on micropatterned or non-micropatterned surface, in hydrogels, or on microcarriers. Stem cells and the cells differentiated from them into specific tissue lineages can be cultured and/or differentiated on dishes with immobilized external stimulus-responsive polymers. Cells can be detached from these dishes without using an enzymatic digestion method or a mechanical method when the appropriate external stimulus is generated on the surface. This review discusses the polymers and polymeric designs employed to produce surface and hydrogels for stem cell culture, differentiation, and/or cell detachment using various external stimuli. 相似文献
A modeling procedure that predicts trihalomethane (THM) formation from field sampling at the treatment plant and along its distribution system using Tampin district, Negeri Sembilan and Sabak Bernam district, Selangor as sources of data were studied and developed. Using Pearson method of correlation, the organic matter measured as TOC showed a positive correlation with formation of THM (r=0.380,P=0.0001 for Tampin and r=0.478,P=0.0001 for Sabak Bernam). Similar positive correlation was also obtained for pH in both districts with Tampin (r=0.362,P=0.0010) and Sabak Bernam (r=0.215,P=0.0010). Chlorine dosage was also found to have low correlation with formation of THM for the two districts with Tampin (r=0.233,P=0.0230) and Sabak Bernam (r=0.505,P=0.0001). Distance from treatment plant was found to have correlation with formation of THM for Tampin district with r=0.353 and P=0.0010. Other parameters such as turbidity, ammonia, temperature and residue chlorine were found to have no correlation with formation of THM. Linear and non-linear models were developed for these two districts. The results obtained were validated using three different sets of field data obtained from own source and district of Seremban (Pantai and Sg. Terip), Negeri Sembilan. Validation results indicated that there was significant difference in the predictive and determined values of THM when two sets of data from districts of Seremban were used with an exception of field data of Sg. Terip for non-linear model developed for district of Tampin. It was found that a non-linear model is slightly better than linear model in terms of percentage prediction errors. The models developed were site specific and the predictive capabilities in the distribution systems vary with different environmental conditions. 相似文献